• ModelCalibration

Goals

The fundamental goal of the Model Calibration phase is the calibration of initial working knee models of each team, which would be the outcomes of Model Development, the previous modeling and simulation phase.

Specific goals of the Model Calibration phase are

• To calibrate two initial knee models
  • one using Open Knee(s) data
  • one using Natural Knee data
• To conduct simulations with models using
  • the same sample simulation case used in Model Development
  • load cases used for calibration that are selected from earmarked data
  • all load cases of earmarked data

• To document and disseminate modeling & simulation (M&S) processes, including specifications and protocol deviations

• To document and disseminate calibration load cases, calibrated parameters and calibrations results.
• To document and disseminate calibrated models including changes in virtual anatomical and mechanical representations of model components
• To document and disseminate simulation results

Tentative Timeline

Earmarked Data

oks003 from Open Knee(s):

• Demographics
  • Left knee
  • Age: 25 years
  • Gender: Female
  • Height: 1.73 m
  • Weight: 68 kg
  • BMI: 22.8
• Specimen-specific mechanical testing data sets
  • 003_optimized passive flexion_0-90_main_processed.tdms - Passive flexion up to 90 degrees (National Instruments TDMS format)
  • 006_All Laxity 0deg_main_processed.tdms - Laxity at 0 degrees flexion (anterior-posterior, internal-external rotation, varus-valgus) (National Instruments TDMS format)

  • 009_AllLaxity 30deg_main_processed.tdms - Laxity at 30 degrees flexion (anterior-posterior, internal-external rotation, varus-valgus) (National Instruments TDMS format)

  • 012_AllLaxity 60deg_main_processed.tdms - Laxity at 60 degrees flexion (anterior-posterior, internal-external rotation, varus-valgus) (National Instruments TDMS format)

  • 014_AllLaxity 90deg_main_processed.tdms - Laxity at 90 degrees flexion (anterior-posterior, internal-external rotation, varus-valgus) (National Instruments TDMS format)

• Other
  • State.cfg - probed points on registration markers and anatomical landmarks; coordinate system transformations; coordinate system offsets (plain text in Microsoft Windows INI style)

  • Any data disseminated for ModelDevelopment including imaging data sets containing registration markers

• Download data in the downloads section of project site, https://simtk.org/projects/kneehub

  • Package name: Data for MC - oks003

  • Link to package location: https://simtk.org/frs/?group_id=1061

  • File name: data-MC-oks003.zip

  • Direct link to file: https://simtk.org/frs/download_confirm.php/file/5637/data-MC-oks003.zip?group_id=1061

DU02 from Natural Knee Data:

• Demographics
  • Right knee
  • Age: 44 years
  • Gender: Male
  • Height: 1.83 m
  • Weight: 70.31 kg
  • BMI: 21.02
• Specimen-specific mechanical testing data sets
  • DU02_INTACT_KE_AP.csv - Anterior-posterior laxity at flexion up to 120 degrees (plain text in CSV format)
  • DU02_INTACT_KE_IE.csv - Internal-external rotation laxity at flexion up to 120 degrees (plain text in CSV format)
  • DU02_INTACT_KE_Passive.csv - Passive flexion up to 120 degrees (plain text in CSV format)
  • DU02_INTACT_KE_VV.csv - Varus-valgus laxity at flexion up to 120 degrees (plain text in CSV format)
• Other

  • Any data disseminated for ModelDevelopment including probe data.

• Download data in the downloads section of project site, https://simtk.org/projects/kneehub

  • Package name: Data for MC - DU02

  • Link to package location: https://simtk.org/frs/?group_id=1061

  • File name: data-MC-DU02.zip

  • Direct link to file: https://simtk.org/frs/download_confirm.php/file/5634/data-MC-DU02.zip?group_id=1061

The modeling teams do not have to use all of the earmarked data. They can also utilize information in literature to define model components that may not have any associated specimen-specific data.

Simulation Cases

Each team will simulate the same simulation case used in ModelDevelopment - passive flexion from 0 to 90 degrees, using the calibrated models. The teams are expected to be faithful to their original interpretation of the term "passive flexion", and therefore its reflection as loading and boundary conditions. The simulations' primary aim will be the prediction of joint kinematics.

Naturally, each team will also simulate the load case(s) that they choose from the earmarked data for calibration. Selection and interpretation of the load case(s) for calibration will be left to individual teams. The simulation(s) will portray joint kinematics-kinetics response.

Each team will also simulate all load cases of the earmarked data for calibration with the calibrated models. Interpretation of these load cases, i.e., their reflection as loading and boundary conditions, will be left to individual teams. The simulation(s) will portray joint kinematics-kinetics response.

Deliverables

Modeling & Simulation Outputs

Intermediate and End-Point M&S Outputs

Two packages representing final and intermediate outcomes of Model Calibration phase will be the outputs of this phase. The focus is the complete curation of the processes and results of each individual team's Model Calibration activities.

• Package (*.zip) collating M&S Outputs using Open Knee(s)

  • End-point M&S Outputs such as

    • Working model including simulation cases in a format preferred by modeling team
    • Simulation results in a format preferred by modeling team

  • Intermediate M&S Outputs in formats preferred by modeling team

    • For each calibration stage
      • Load case(s) used for calibration simulations
      • Fitted data
      • Calibrated parameters (before and after)
      • Calibration fit error (before and after)
      • Virtual representations of model components that are changed during calibration (before and after)
    • For every load case of earmarked data
      • Coordinate systems
      • Extracted joint kinematics-kinetics used for simulation
      • Loading and boundary conditions to represent loading case in simulations

• Package (*.zip) collating M&S Outputs using Natural Knee Data

  • Similar to package described above but utilizing Natural Knee Data

Documentation of M&S Outputs

Two document sets should provide information to help navigate the M&S output package and to describe what M&S Outputs are (not how they are obtained).

• Document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Outputs using Open Knee(s) data

  • Metadata for the document - authors, date, contact information
  • Information to navigate the package

  • Information to describe what each M&S Output is

• Document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Outputs using Natural Knee Data

  • Similar to document described above but utilizing Natural Knee Data

Modeling & Simulation Processes

An important set of deliverables are the documentation of the M&S processes, i.e. the individualized workflows to transform earmarked data to M&S outputs. The documents will include Specifications, to provide each team's initial plans for processing data to obtain M&S outputs, and Protocol Deviations, to describe modifications to Specifications during the execution of the initial plans. Ideally, when a third-party modeler has the earmarked data, specifications, and protocol deviations, they can follow the steps of the modeling team to reproduce their disseminated M&S outputs.

Specifications

Specifications documents are aimed to describe individual steps to obtain M&S outputs from earmarked data. The detail should be provided in a manner that other teams can reproduce the decision making process of the modeler. Any subjective and arbitrary decisions should be noted and as much insight as possible should be provided for justification and reproduction these decisions. This document should be prepared and disseminated before M&S activities start.

• Specifications document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Processes using Open Knee(s) data

  • Metadata for the document - authors, date, contact information

  • Information to describe what each target M&S Output is

  • Information to describe how each target M&S Output will be obtained

  • Anticipated burden of the workflow

• Specifications document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Processes using Natural Knee Data

  • Similar to document described above but utilizing Natural Knee Data

Protocol Deviations

If specifications need to be updated during activities to go from earmarked data to M&S outputs, these should be documented as protocol deviations. As much insight as possible should be provided for justification of the deviation. This document should be prepared during M&S activities and disseminated along with M&S outputs.

• Protocol Deviation document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Processes using Open Knee(s) data

  • Metadata for the document - authors, date, contact information
  • Collated set of protocol deviations, each describing
    • What is the deviation in relevance to Specifications?

    • Which M&S output does the deviation impact?

    • Who implemented the modification?
    • When was the modification made?
    • What is the justification for deviation?
  • Actual burden of the workflow

• Protocol Deviation document (in source, e.g., *.odt, *.docx, etc., and PDF formats) describing M&S Processes using Natural Knee Data

  • Similar to document described above but utilizing Natural Knee Data

Package for Comparative Analysis

Two packages will be prepared to curate M&S Outputs and process documentation of each team; one starting with data from Open Knee(s), the other utilizing Natural Knee data. The M&S Output files submitted by the individual teams will be organized in these packages and provided as is with the possible exception of renaming of master package files. The documents will be provided as is. Yet, additional organization of output description and process documentation may be necessary to meet third-party requirements or recommendations (see guidance from #USFDA, #ASME, #NASA, #Erdemir2015).

• Package of M&S outputs and processes using data from Open Knee(s)

  • Cleveland Clinic team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • University of Denver team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Cleveland State University team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Auckland Bioengineering Institute team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Hospital for Special Surgery team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

• Package of M&S outputs and processes using Natural Knee data

  • Cleveland Clinic team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • University of Denver team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Cleveland State University team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Auckland Bioengineering Institute team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

  • Hospital for Special Surgery team

    • M&S outputs package file

    • M&S outputs documentation files

    • M&S processes specifications documents

    • M&S processes protocol deviation documents

Guidance

Where to get the Earmarked Data?

The teams can download data in the downloads section of project site, https://simtk.org/projects/kneehub, check packages

• Data for Model Calibration - oks003, from Open Knee(s)
• Data for Model Calibration - DU02, from Natural Knee Data

Tell me more about joint mechanics data from Natural Knee Data

Mechanical testing of the Natural Knee Data is described in Harris et al. (2016). In relation to earmarked data, the loads representative of passive flexion and laxity were manually applied with live feedback on loading. The data were processed and described in a joint coordinate system. The data files are in human readable CSV format, including kinematics-kinetics of the tibiofemoral joint in an anatomical coordinate system representation. Files represent separate loading cases of laxity or passive flexion.

Known Limitations

This list provides identified limitations that needs to be managed by all teams while processing the data for modeling and simulation.

• Time synchronization of load and displacement data may not be ideal. The teams may need to temporally adjust data for appropriate alignment.

Clarifications

This list provides more information to clarify descriptions of data for modeling and simulation.

• DU02_tib_GSPts.txt - The order and the definition of anatomical landmarks were ambiguous in reference to Harris et al. (2016). These were clarified by University of Denver team as:

  • Medial Plateau Dwell
  • Lateral Plateau Dwell
  • Medial Spine of Tibial Eminence
  • Center of Tibial Eminence (origin of the tibial coordinate system)
  • Center of Distal Intermedullary Canal
• The kinematics data are represented in the joint coordinate system, i.e. the rotations are about the three orthogonal axes (α,β,γ), and the translations are about the tibial local coordinate system (q1,q2,q3) as described by Grood and Suntay (1983) with the exception of the superior-inferior DOF which has been negated to keep translations positive w.r.t the tibial local coordinate system. Specifically kinematics adhere to the following convention: +flexion, +valgus, +external tibia, +lateral tibia, +anterior tibia, +superior tibia.
• The kinetics data are represented in the tibia local coordinate system (Grood and Suntay 1983 - fig 2). There is an ambiguity in the definition of reported loads in the calibration data, e.g. reaction loads or applied loads. This may need to be resolved by all teams in whatever way they may consider as appropriate.

Tell me more about joint mechanics data from Open Knee(s)

Mechanical testing of Open Knee(s) is described in specifications at #OKS-SEJM. Notes related to experimentation on the specimen can be found at #OKS-oks003. The Open Knee(s) data is in a raw form, i.e. as collected using the robotics testing system. It may be useful to get familiar with the tools and experimentation setup to navigate the contents of the data files. A cohort of documents describing testing infrastructure, data collection and storage conventions is available at #OKS-IEM.

Coordinate system conventions used for data collection and storage are described in detail at #OKS-KCS. In a nutshell, kinematics-kinetics data are represented in anatomical joint coordinate systems. There are some caveats that the users of the data should pay attention to. First, there are two anatomical coordinate systems - an original based on anatomical landmarks, and an optimized to minimize off-axis motion representation during passive flexion. The latter only changes the original femur coordinate system to re-orient the flexion axis. Second, the joint coordinate systems are in right knee abstraction, i.e. essentially a left knee, if tested, is mirrored for convenience in testing. This situation needs to be accommodated when registering the data to physical coordinate systems, i.e that of imaging. Third, the movements (translations and rotations) are described along the axis of joint coordinate system; a femur fixed flexion axis, a tibia fixed internal-external rotation axis, and a floating axis. Fourth, the kinematics are represented relative to a reference testing state of the joint, which is recorded in configuration file as an offset. Last, the joint loads are represented in tibia coordinate system with a clinical convention, i.e. external loads applied to the tibia as in a physical exam.

Two types of files are provided with the Open Knee(s) data set. A configuration file (.cfg) contains settings and data necessary to perform mechanical testing. Contents of this file are described in detail at #OKS-SCF. Of importance to model calibration are content related to coordinate system transformations between motion tracking sensors (femur and tibia as rigid bodies), robot and motion tracking system (worlds), anatomical coordinate systems (original and optimized, in right knee abstraction); joint coordinate system offsets (representing reference kinematics of the joint) probed points (anatomical and registration markers) including the point in world coordinate system and rigid body pose and orientation (where the point rests) in world coordinate system.

The second type of file contains mechanical data in a National Instruments file format (.tdms). This file is binary but can be accessed with widely available tools for Python and Matlab. Passive flexion data are provided along with laxity data. For laxity testing, each file represents mechanics of the joint at a fixed flexion angle (based on optimized joint coordinate system) focusing on testing of three degrees of freedom one at a time: anterior-posterior, varus-valgus, and internal-external rotation. Raw motion data are stored in relevant channels, e.g. pose and orientation of femur and tibia motion sensors (Sensor.Femur sensor and Sensor.Tibia Sensor), loads in load cell coordinate system (Sensor.Static Load Cell). During data collection, the raw mechanics data are processed to facilitate control. The processed data is in a right knee abstraction, i.e. left knees are represented in a mirrored fashion for testing. The processed kinematics-kinetics data include joint kinematics: in original anatomical coordinate system (State.Knee JCS) or optimized one (State.JCS) (all relative to reference/offset state in configuration file); and joint kinetics: in tibia coordinate system (State.JCS Load) with loads on tibia. It should be noted that the tibia coordinate system of original and optimized anatomical coordinate systems are the same, only the femur coordinate system is reoriented (optimized) to minimize off-axis motions of passive flexion. For more details of file contents, one can refer to #OKS-TDMS.

Known Limitations

This list provides identified limitations that needs to be managed by all teams while processing the data for modeling and simulation.

Clarifications

This list provides more information to clarify descriptions of data for modeling and simulation.

How to prepare and submit Specifications?

Specifications are concerned about documentation of the processes to transform earmarked data to M&S outputs. This information is intended to be provided separately for processing of both data sets to capture nuances in response to variable data content and quality. This documentation will be provided before execution of Model Calibration activities to demonstrate the path each modeling team has chosen. Specifications should be in a detail such that other modeling teams can follow the steps to reproduce M&S outputs. When possible, justifications to support modeling & simulation decisions should be provided.

Recommended content for Specifications includes:

• Metadata for the document - contributors, contact information for corresponding author, date of document.

• Summary of data utilized with the M&S processes.

• Overview of target M&S outputs and M&S processes - similar to summarizing it in a scholarly publication.

• Detailed descriptions of target M&S outputs (consult Erdemir et al. (2012), for completeness)

  • Complete calibrated model
    • Anatomy
    • Mechanical properties
    • Coordinate systems
    • Loading and boundary conditions (for the simulation cases)
    • Interactions between model components (contacts, ties)
  • Intermediate and final calibration outcomes
    • Calibration data
    • Calibrated parameters
    • Calibration fit error
    • Changed model components (anatomical and/or mechanical)
  • Intermediate and final outcomes of analysis of experimental load cases
    • Source data (as extracted from earmarked data set)
    • Processed data (as analyzed to make ready for use in simulations)
  • Simulation cases
    • Loading and boundary conditions
    • Target metrics for predictions
    • Numerical analysis settings
    • Anticipated results

• Detailed descriptions of M&S processes to get the target M&S outputs

  • Steps to calibrate models
  • Steps to implement load cases of earmarked data as loading and boundary conditions
  • Steps to perform simulations
  • Burden
    • Software requirements
    • Hardware requirements
    • Anticipated man hours and expertise level
    • Computational cost
• References

Each team will submit two sets of documents, one for Model Calibration using Open Knee(s) data and another for Model Calibration using Natural Knee data. Each set will include the source document, i.e., in preferred format that it is written, and as a PDF copy. The documents should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted documents to the project site in the Documents section with labeling indicative of modeling phase, modeling teams, and data source.

How to prepare and submit Protocol Deviations?

It is likely that initial specifications may need to be modified to accomplish Model Calibration activities. These modifications, i.e. Protocol Deviations, should be documented during Model Calibration activities. Each protocol deviation should include information to understand who did, what and when, and why. The documentation should be in a detail such that other modeling teams can utilize it along with specifications to reproduce M&S outputs. This document should also include the actual burden of M&S activities including changes in software and hardware requirements, and deviations from anticipated labor effort and computational cost. Protocol Deviations should be collated in two sets of documents (one for using Open Knee(s) data and another for Natural Knee data). The documents will be provided after execution of Model Calibration activities in source form, i.e., in preferred format that it is written, and as a PDF copy. The documents should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted documents to the project site in the Documents section with labeling indicative of modeling phase, modeling teams, and data source.

How to curate, document and submit Modeling and Simulation Outputs?

All M&S outputs (intermediate and final) should be provided in their preferred format, i.e., accommodating individual teams' workflows. Each team should provide two packages (one for Open Knee(s) data and another for Natural Knee data) as a compressed archive (e.g. *.zip). For each package, an accompanying document should describe M&S outputs - in source form, i.e., in preferred format that it is written, and as a PDF copy. This information should focus on i) file descriptors and formats, and folder organization to help navigate the provided content and ii) what actually each M&S output is (NOT how they are obtained). The latter should be reflective of the description of target intermediate and final M&S products of Specifications. It should also be indicative of necessary information for other modeling teams to judge the nature and extent of the provided content. The packages should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted packages to the project site in the Downloads section with labeling indicative of modeling phase, modeling teams, and data source.

Preparing a package for Comparative Analysis

All material will likely need to be reorganized and converted to respond to third-party requirements for comparative analysis (see guidance from #USFDA, #ASME, #NASA, #Erdemir2015). Nonetheless, this should not prevent individual M&S teams to personalize their workflows and documentation style. Preparation of a package for comparative analysis will collate M&S outputs and documents submitted by the individual teams. M&S outputs will be provided in formats as delivered by the teams. Documentation may need to be reorganized using source copies of documentation from individual teams. Ahmet Erdemir, erdemira@ccf.org, will submit the prepared package to the third-party by any convenient means - e-mail, file sharing, upload to project site, etc.

References

Erdemir, A., Mulugeta, L. and Lytton, W. W. Ten “not so” simple rules for credible practice of modeling and simulation in healthcare: a multidisciplinary committee perspective, 2015 Biomedical Engineering Society / Food and Drug Administration Frontiers in Medical Devices Conference: Innovations in Modeling and Simulation, May 18-20, 2015, Washington, DC. Available at: https://simtk.org/svn/cpms/doc/posters/fmd_2015.pdf.

Erdemir, A., Guess, T. M., Halloran, J. P., Tadepalli, S. C. and Morrison, T. M. (2012) Considerations for reporting finite element analysis studies in biomechanics, Journal of Biomechanics, 45, 625-633. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278509/.

U.S. Food and Drug Administration. Reporting of Computational Modeling Studies in Medical Device Submissions. Available at: https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM381813.

NASA-STD-7009: Standard for Models and Simulations. Available at: https://standards.nasa.gov/standard/nasa/nasa-std-7009.

ASME V&V American Society of Mechanical Engineers - Committee Pages - V&V 40 Verification and Validation in Computational Modeling of Medical Devices. Available at: https://cstools.asme.org/csconnect/CommitteePages.cfm?Committee=100108782.

Harris, M. D., Cyr, A. J., Ali, A. A., Fitzpatrick, C. K., Rullkoetter, P. J., Maletsky, L. P. and Shelburne, K. B. (2016) A combined experimental and computational approach to subject-specific analysis of knee joint laxity, Journal of Biomechanical Engineering, 138, 081004. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4967880/.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - Specifications/ExperimentationJointMechanics. Available at: https://simtk.org/plugins/moinmoin/openknee/Specifications/ExperimentationJointMechanics.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - oks003. Available at: https://simtk.org/plugins/moinmoin/openknee/oks003.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - Infrastructure/ExperimentationMechanics. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics.

Knee Joint Coordinate System v2.1. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=Knee+Coordinate+Systems.pdf.

bioRobotics, simVitro - State Configuration File Explanations, User Manual, 2013CB-031-001.A. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=2013CB-031-001.A_State+Configuration+File+Explanations.pdf.

bioRobotics, simVitro - simVITRO Data File Contents for Open Knee, User Manual, 2013CB-031-002.B. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=2013CB-031-002.B+simVITRO+Data+File+Contents_Open+Knee.pdf.

Footnotes